Various types of devices are used to decompress, decode and/or otherwise reproduce video programming content. A conventional set top box (STB), for example, typically decodes video signals received from a cable, direct broadcast satellite (DBS) or other broadcast source for presentation to a viewer on a television or other display. Other devices that provide video reproduction include other types of broadcast television receivers, as well as televisions and video displays themselves, video cards, video processing circuitry, computers, and many others.
Generally, it is desirable to verify that manufactured video reproduction devices are functioning properly during before the devices are shipped to the customer or distributor. Often, human subjective testing is used to evaluate video encoding and decoding robustness during the design process, and to ensure quality and proper functioning after manufacturing. Particularly in the manufacturing arena, human subjective testing can be relatively labor intensive and expensive. Moreover, human testing is inherently susceptible to variations in subjective human review standards, thereby potentially leading to a range of quality fluctuations that can be quite difficult to normalize or control.
Although several types of automated testing schemes have been developed, each of the known techniques exhibits one or more disadvantages in practice. One type of automatic picture quality analysis, for example, uses machine vision to capture a portion of a pre-recorded video test sequence that is displayed in response to the system under test. The captured portion of the display is analyzed using digital signal processing hardware, and a more objective (e.g., numeric) “picture quality rating” is produced based upon spatial, temporal, color and/or other analysis. Problems can arise, however, with straightforward comparisons of test and reference sequences to generate the quality metrics. Spatial or temporal misalignments between test and reference sequences, for example, can greatly affect such measurements, leading to inaccuracies. Further, temporal artifacts (e.g., repeated frames taking the places of lost original frames) can occur due to transmission errors, buffer overflow or underflow, or other factors, thereby interfering with the results of the analysis. Other issues could also arise, leading to uncertainty, inaccuracy and/or variation in the “objective” metric determined by a machine-vision system.
As a result, it is desirable to create systems, methods and/or devices that are able to effectively yet efficiently test video decoding hardware such as set top boxes, video receivers and/or the like. Various features and characteristics will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and this background section.